Information processing apparatus having a contact detection unit capable of detecting a plurality of contact points, storage medium having program recorded thereon, and object movement method

ABSTRACT

An information processing apparatus has a display unit operable to display an object, a contact detection unit operable to detect a plurality of contact points on the display unit, and a control unit operable to control the object displayed on the display unit so as to be movable with use of the contact detection unit. When the control unit detects contact with a display range of the object displayed on the display unit with use of the contact detection unit, detects contact with an active range of a display screen other than the display range of the object, and further detects a movement instruction to the object, then the control unit controls the object so as to be movable within the active range. Thus, use of a contact detection unit capable of a plurality of contact points permits a user&#39;s intuitive operation.

TECHNICAL FIELD

The present invention relates to an information processing apparatushaving a contact detection means capable of detecting a plurality ofcontact points, a storage medium on which a program used in theinformation processing apparatus is recorded, and an object movementmethod.

BACKGROUND ART

A contact detection unit called a touch panel or a touch screen isprovided on various information processing apparatuses these days.Furthermore, a contact detection means capable of detecting a pluralityof contact points has been developed instead of a conventional contactdetection means capable of detecting only one contact point.

Japanese laid-open patent publications Nos. 2002-304256 and 2007-279638disclose a touch panel display or a touch panel device capable ofdetecting a plurality of contact points.

DISCLOSURE OF INVENTION

An operation method of an information processing apparatus having acontact detection means is designed so as to allow a user to intuitivelyoperate the information processing apparatus. While such an operationmethod provides a user with intuitive operation, an excessive burden isimposed on the user if the operation (command to a controller,transition of a screen, or the like) is not performed as expected by theuser' intuitive image.

For example, an information processing apparatus that scrolls displayedcontents by detection of contact with a touch panel (determination bytouch) or by detection of removal of contact with the touch panel(determination by touch and release) provides a user with sufficientoperability on a scroll operation. However, in a case of detailedoperations, such as a coordinate movement of only a desired part ofdisplayed information, a scroll operation of only a desired part ofdisplayed information, or an operation using a plurality of contactpoints, a (slight) difference is likely to be produced between thesetting of the information processing apparatus and the user's intuitiveoperation. In such a case, the user will repeat intuitive operations.For example, even though a user attempts a scroll operation and touchesa touch panel, the information processing apparatus does not operate ifa scroll operation is set to be activated through the determination bytouch and release. In such a case, the user repeats the same operationbecause no desired operation is performed. Thus, an excessive burden isimposed on the user.

As a result, a command to a controller or transition of a screen (suchas scrolling) that has not been intended by the user is provided.Therefore, sufficient operability cannot be provided.

The present invention has been made in view of the above drawbacks. Thepresent invention provides an information processing apparatus that canproperly detect a user's intuitive operation with use of a contactdetection unit capable of detecting a plurality of contact points andcan properly move an object.

An information processing apparatus according to the present inventionincludes a display unit operable to display an object, a contactdetection unit operable to detect a plurality of contact points on adisplay screen of the display unit, and a control unit operable tocontrol the object displayed on the display unit so as to be movablewith use of the contact detection unit. When the control unit detects afirst contact point in an active range of the display screen other thana display range of the object with use of the contact detection unit,detects a second contact point in the display range of the object, andfurther detects a movement instruction to the object, then the controlunit controls the object so as to be movable within the active range.

According to the present invention, there can be provided an informationprocessing apparatus that can properly detect a user's intuitiveoperation with use of a contact detection unit capable of detecting aplurality of contact points and can properly move an object.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a functional block diagram showing an information processingapparatus according to a first embodiment of the present invention.

FIG. 2 is a flow chart explanatory of an identification operation of theinformation processing apparatus of FIG. 1.

FIG. 3 is a functional block diagram showing a mobile terminal as aninformation processing apparatus according to a second embodiment of thepresent invention.

FIG. 4 is a flow chart explanatory of a scroll operation of the mobileterminal of FIG. 3.

FIGS. 5A and 5B are diagrams showing a transition example of a displayscreen in the mobile terminal of FIG. 3 in a case of a single touch.

FIGS. 6A and 6B are diagrams showing a first transition example of thedisplay screen in the mobile terminal of FIG. 3 in a case of multipletouches.

FIGS. 7A and 7B are diagrams showing a second transition example of thedisplay screen in the mobile terminal of FIG. 3 in a case of multipletouches.

FIGS. 8A and 8B are diagrams showing a third transition example of thedisplay screen in the mobile terminal of FIG. 3 in a case of multipletouches.

FIGS. 9A and 9B are diagrams showing a fourth transition example of thedisplay screen in the mobile terminal of FIG. 3 in a case of multipletouches.

FIGS. 10A and 10B are diagrams showing a fifth transition example of thedisplay screen in the mobile terminal of FIG. 3 in a case of multipletouches.

FIGS. 11A and 11B are diagrams showing a sixth transition example of thedisplay screen in the mobile terminal of FIG. 3 in a case of multipletouches.

FIG. 12 is a partial enlarged view of the display screen shown in FIG.11.

FIG. 13 is a functional block diagram showing a tablet computer as aninformation processing apparatus according to a third embodiment of thepresent invention.

FIG. 14 is a flow chart explanatory of a movement operation of thetablet computer of FIG. 13.

FIGS. 15A and 15B are diagrams showing a transition example of a displayscreen of a spreadsheet program in the tablet computer of FIG. 13 in acase of multiple touches.

BEST MODE FOR CARRYING OUT THE INVENTION

An information processing apparatus according to a first embodiment ofthe present invention will be described with reference to FIGS. 1 and 2.For example, an information processing apparatus according to thepresent invention is operated by a program control and is formed as amobile terminal having a communication function (telephone conversation,mail transmission and reception, and Internet connection), a calendarfunction, a scheduler function, and the like. Other examples of theinformation processing apparatus include a personal computer, a smartphone, a PDA (Personal Digital Assistants), and a PHS (PersonalHandyphone System) having a touch screen function.

FIG. 1 is a functional block diagram showing an overview of aconfiguration of an information processing apparatus 100 according tothe first embodiment.

The information processing apparatus 100 shown in FIG. 1 includes acontrol unit 110 operable to perform a variety of calculations, adisplay unit 120 operable to display information directed from thecontrol unit 110, a contact detection unit 121 operable to detect thepresence of a contact point in a coordinate area of the display unit 110in which the display information is displayed, a coordinate of thecontact point (contact coordinate or detection coordinate), and a change(movement) of the detection coordinate, and a storage unit 130 formed bya ROM, a RAM, a HDD, a flash memory, or the like. The storage unit 130stores therein an operating system, a driver or an application programfor the contact detection unit 121, various types of setting informationof the information processing apparatus 100, and the like.

The control unit 110 is operable to control the display unit 120 so asto display first display information in a first display range.Furthermore, the control unit 110 is operable to display second displayinformation in a second display range that is part of the first displayrange. Here, the term “display range” refers to an area (outer frame) inwhich display information can be displayed. The term “coordinate area”refers to an area in which display information has actually beendisplayed. While the second display range is included in the firstdisplay range, a coordinate area in which the second display informationhas been displayed is not included in a coordinate area in which thefirst display information has been displayed. Furthermore, the controlunit 110 can detect, via the contact detection unit 121, the presence ofa contact point in a coordinate area of the display unit 120 in whichthe first and second display information has been displayed. The controlunit 110 can also identify a change (movement) of the detectedcoordinate (detection coordinate). Moreover, the control unit 110 candetect a plurality of contact points in a coordinate area in which thefirst display information is displayed and in a coordinate area in whichthe second display information is displayed. Furthermore, when a changeof a detection coordinate of only one of a plurality of contact pointsis detected, the control unit 110 performs a process of moving(scrolling) only the display information (object) displayed in acoordinate area in which the change of the detection coordinate of thecontact point has been detected based upon the change of the detectioncoordinate.

The first display information, which is displayed in the first displayrange, which is the entire display screen or a predetermined area(active range), and the second display information, which is displayedin part of that display range, are displayed on the display unit 120 bythe control unit 110.

The second display information (object) may include an image, a figure,a character, or a symbol, or a combination thereof. Furthermore, thesecond display information may be expressed by any one of an icon, abutton, a character string, a text box, and area information visuallyindicated for providing a command to the control unit 110, or acombination thereof. Moreover, the second display information includinga character string also includes information for managing a selectedcharacter string as a bundle after the character string has beenselected by range specification.

With this configuration, the information processing apparatus 100 canproperly detect a user's intuitive operation with use of the contactdetection unit 121 capable of detecting a plurality of contact pointsand can properly move an object. In this specification, the term“contact point” includes not only a relatively small “point” as in acase where a stylus pen or the like is used, but also a relatively large“area” as in a case where a human finger or the like is used.

Next, operation of the information processing apparatus 100 will bedescribed.

In accordance with the application program and the operating systemrecorded in the storage unit 130, the control unit 110 of theinformation processing apparatus 100 produces images of the firstdisplay information to be displayed in the first display range and thesecond display information to be displayed in the first display range ofthe screen of the display unit 120 and then displays them on the displayunit 120. Concurrently, in accordance with the driver for the contactdetection unit 121, the operating system and application program, thecontrol unit 110 detects contact with the first coordinate area in whichthe first display information has been displayed and the secondcoordinate area in which the second display information has beendisplayed (appearance of a contact point), removal of contact(disappearance of the contact point), detection coordinates, and changesof the detection coordinates with use of the contact detection unit 121.The control unit 110 recognizes the detected contact, removal ofcontact, and changes of the detection coordinates as determination bytouch, determination by touch and release, or a movement instruction asneeded and determines an operational instruction to the displayed objector the like.

There will be described a process of the control unit 110 thatidentifies a movement instruction of the object from changes ofdetection coordinates of multiple contact points detected with thecontact detection unit 121.

In accordance with various types of software programs, the control unit110 detects a plurality of contact points with the contact detectionunit 121 and identifies a user's movement instruction based upon theposition coordinates (detection coordinates) of the contact points andthe changes of those position coordinates. A movement instruction can bedistinguished from other instructions by vectorizing detectioncoordinates of individual contact points so as to identify movementdirections. Furthermore, the movement instruction may be distinguishedfrom other instructions by obtaining a period of time from the detectionof contact to the detection of removal of contact with respect to eachcontact point and relying upon differences of those periods.

For example, when two contact points are concurrently detected, anddetection coordinates of those contact points move concurrently, thenthe operation can be determined to be magnification or reduction of thedisplay screen. Meanwhile, when two contact points are detected at aninterval of time, and a detection coordinate of one of those contactpoints hardly moves, and a detection coordinate of the other contactpoint moves, then the operation can be determined to be a movementinstruction. With such an identifying process, different operations canbe performed depending upon a difference in change of detectioncoordinates of the multiple contact points.

Furthermore, operations may be differentiated by using the size of acontact area. For example, operations may be differentiated by using adifference in size of a contact area between a stylus pen and a finger.Moreover, operations may be differentiated by using a difference in sizeof a contact area between a thumb and other fingers. For example, thecontact detection unit 121 may be set such that an operation of only anunderlay of the display screen or only a fixed operation is allowedaround an area at which a finger to hold the information processingapparatus 100 is located. Furthermore, depending upon the size of thedisplay information in the display screen, a contact point having alarge contact area or a small contact area may be determined to beinvalid. Additionally, operations may be differentiated depending uponwhether the information processing apparatus 100 is held by a singlehand or by both hands.

In any case, even if a movement instruction is detected by contact witha coordinate area of one object, the program allows the control unit 110of the information processing apparatus 100 to perform a process ofmoving only one object while fixing the underlay or the background orperform a process of moving the entire active display screen based uponthe positional relationship with contact with other coordinate areas, aduration of contact, and the like. Thus, a user's intuitive operationcan properly be detected.

FIG. 2 is a flow chart explanatory of an identification operation of theinformation processing apparatus 100. The identification process of theoperation shown in FIG. 2 is to provide a function of moving an object,which is provided as a user interface to a user when the aforementionedinformation processing apparatus 100 operates.

The control unit 110 of the information processing apparatus 100monitors contact with the display screen of the display unit 120 via thecontact detection unit 121 (Step S201).

If any contact is detected, the control unit 110 judges whether aplurality of contact points are concurrently present. The processproceeds to the next Step S203 if a plurality of contact points havebeen detected, and the process proceeds to Step S205 if only one contactpoint has been detected (Step S202). In other words, the control unit110 judges whether a contact point has been detected in only one of thefirst and second coordinate areas in which the first and second displayinformation has been displayed, or whether a plurality of contact pointshave been displayed in both of the first and second coordinate areas.Furthermore, in addition to detection of a contact point in the displayrange of the object, the control unit 110 judges whether a contact pointhas been detected in an active range of the display screen other thanthe display range of the object.

Detections of contact points in both of the first coordinate area andthe second coordinate area are judged to be the same input operationwhichever is first detected. Nevertheless, concurrent detections ofcontact points in the first coordinate area and the second coordinatearea may be judged to be different input operations.

When a plurality of contact points have been detected, the control unit110 monitors detection coordinates of those detected contact points. Thecontrol unit 110 judges whether only one of the contact points has moved(a movement (scroll) instruction has been provided). The processproceeds to the next Step S204 if only one contact point has moved, andthe process proceeds to the next Step S206 if the contact points havemoved concurrently (Step S203). In other words, the control unit 110judges whether a change of a detection coordinate of only one of thecontact points has been detected. Saying it differently, the controlunit 110 detects a contact point in an active range of the displayscreen other than the display range of the object to be moved and judgeswhether a movement instruction to the object has been detected.

If another operation (instruction) that is different from the movementinstruction is made, the control unit 110 performs a processcorresponding to that operation.

If movement of only one contact point has been detected, the controlunit 110 fixes, on the display screen, the display information displayedin the coordinate area in which the other contact point is present, andmoves the display information displayed in the coordinate area in whichmovement of the contact point has been detected (Step S204). In otherwords, only the display information displayed in the coordinate area inwhich a change of the detection coordinate of the contact point has beendetected is moved based upon the change of the detection coordinate.Saying it differently, an object is controlled so as to be movablewithin the active range, and an object to which a movement instructionhas been provided is moved.

In the case of one contact point, the control unit 110 monitors thedetection coordinate of the detected contact point and judges whether amovement instruction has been provided by a user (Step S205).

If the detected contact points concurrently move in the same direction,or if one contact point has been detected and the detection coordinatethereof changes, the control unit 110 moves (scrolls) the entire activedisplay information (Step S206). In other words, if changes of thedetection coordinates of all of the contact points have been detected,then the control unit 110 moves the first and second display informationbased upon the changes of the detection coordinates.

With the above identification operation, the information processingapparatus 100 can properly detect a user's intuitive operation with useof a contact detection unit capable of detecting a plurality of contactpoints and can thus move display information (object).

Next, a mobile terminal 200 will be described as an informationprocessing apparatus according to a second embodiment of the presentinvention.

FIG. 3 is a functional block diagram showing an overview of aconfiguration of the mobile terminal 200 according to the presentembodiment.

As shown in FIG. 3, the mobile terminal 200 includes a radio unit 201, abroadcast receiver unit 202, a GPS receiver unit 203, a camera unit 204,an acoustical treatment unit 205, a display unit 206 having a contactdetection function, a storage unit 207, and a control unit 208.

The radio unit 201 is operable to wirelessly transmit information to andreceive information from a radio base station apparatus via an antenna.The broadcast receiver unit 202 is operable to receive broadcast signalstransmitted from a broadcast station (a ground station, a satellite, orthe like) and conducts signal processing of visual data, audio data,information data, and the like, which have been obtained by demodulatingthe received broadcast signals. The GPS receiver unit 203 is operable tomeasure periods of time for radio waves emitted from a plurality of GPSsatellites to reach the mobile terminal 200, compute distances betweenthe respective GPS satellites and the mobile terminal 200, and compute aposition coordinate with use of the computed distances. The camera unit204 is operable to acquire image information (take a photograph). Theacoustical treatment unit 205 is operable to process acoustic signals ofmusic, notification sounds, or voice inputted and outputted via amicrophone or a speaker.

The display unit 206 having a contact detection function is a touchscreen (touch panel) having a display function of outputting displayinformation such as images, figures, characters, and symbols (videosignal processing) and a switch function of sensing a user's inputoperation by using pressure, capacitance, or the like (positioncoordinate detection function and contact detection function).

The storage unit 207 stores therein telephone directory information,sent and received mail information, outgoing and incoming callinformation, content information, application programs, photographedimages, various types of setting information of the mobile terminal 200,and the like.

The control unit 208 is operable to control a telephone function, a mailfunction, an Internet (web) connection function, a camera function, atelevision receiving function, a GPS (positioning) function, a contentplayback function, other functions of the terminal, and respectivecomponents of the terminal Furthermore, the control unit 208 is operableto control a display function of the display unit 206 having a contactdetection function and display an electronic screen on the display unit206. The control unit 208 is operable to detect a user's operationalinstruction by using the switch function (position coordinate detectionfunction and contact detection function) of the display unit 206 havinga contact detection function.

Next, an identification process of an operation (instruction) of themobile terminal 200 will be described with reference to FIG. 4.

FIG. 4 is a flow chart explanatory of a scroll operation of the mobileterminal 200.

The control unit 208 of the mobile terminal 200 monitors contact withthe display unit 206 having a contact detection function (Step S401). Ifany contact is detected, the control unit 208 judges whether the numberof contact points is one or more. The process proceeds to the next StepS403 if a plurality of contact points have been detected, and theprocess proceeds to Step S405 if one contact point has been detected(Step S402).

If a plurality of contact points have been detected, the control unit208 judges whether at least one of those contact points is located at apredetermined position (or within an area (movement-fixing coordinatearea)). The predetermined position will be discussed later.

The process proceeds to the next Step S404 if any contact has beendetected at the preset position, and the process proceeds to Step S405if no contact has been detected at the preset position (Step S403).

If the control unit 208 detects any contact at the preset position andalso detects a movement instruction to an object (corresponding to thesecond display information), the control unit 208 fixes the backgrounddisplay (corresponding to the first display information) and controlsthe object so as to be movable within the active range (Step S404).

Meanwhile, if one contact point has been detected, the control unit 208scrolls the entire display based upon the amount of movement of thecontact point until the contact is removed (the contact pointdisappears) (Steps S205-S206).

The preset position (or area) will be described. The preset position isset within a range of the display screen other than an object to bemoved, or within an active range of the display screen other than anobject to be moved, or at a position (designated position) that can becontacted (pressed) by a user's hand to hold the mobile terminal 200.Saying it differently, the preset position is set within the activerange of the display screen other than the display range of the object,or within the first coordinate area in which the first displayinformation has been displayed, other than the second coordinate area inwhich the second display information has been displayed and in which thecontact point has been detected, or within a designated coordinate areadesignated by an application program, or within the display range of amovement-fixed object.

The position that can be contacted (pressed) is a position that can bereached by a thumb or another specified finger when the mobile terminal200 is held in a recommended standard manner. The preset position willbe exemplified in FIG. 7A and figures following FIG. 7A.

Next, some transition examples of the display screen of the mobileterminal 200 will be described. FIGS. 5A to 11B are diagrams showingtransition examples of the display screen of the mobile terminal 200.

FIGS. 5A to 11B show the entire display screen displayed on the displayunit 206 having a contact detection function by the control unit 208.FIGS. 5A, 6A, 7A, 8A, 9A, 10A, and 11A show the display screen prior tomovement (scroll) of an object. On the other hand, FIGS. 5B, 6B, 7B, 8B,9B, 10B, and 11B show the display screen after movement of an object.

FIGS. 5A and 5B are diagrams showing a transition example of the displayscreen in a case of a single touch.

In FIG. 5A, first display information (schedule chart) 501 is displayedin a display screen frame 500, and second display information (object)502 is displayed. In this state, it is assumed that a user touches asecond coordinate area in which the second display information 502 hasbeen displayed with his/her finger and then moves his/her fingerdownward as indicated by the arrow in FIG. 5A. The control unit 208 doesnot detect any contact point in a first coordinate area in which thefirst display information 501 has been displayed, but detects a contactpoint in the second coordinate area in which the second displayinformation 502 has been displayed. The control unit 208 detects achange of a detection coordinate of the contact point as a scrollinstruction. That is, FIG. 5A shows an operation from Step S401 via “No”of Step S402 to Step S405 shown in FIG. 4.

FIG. 5B shows a display result in which the control unit 208 hasscrolled the first and second display information 501 and 502, i.e., theentire display information, based upon the change of the detectioncoordinate as a result of the identification process of FIG. 5A. Thatis, FIG. 5B shows a state after the operation of Step S406 shown in FIG.4.

As shown in FIG. 5A, when the user attempts to move the object in a casewhere a contact point has been detected only in the display range of theobject (the second display information 502), then the schedule chart(first display information 501) and the object are scrolled at the sametime.

FIGS. 6A and 6B are diagrams showing a transition example of the displayscreen in a case of multiple touches. FIGS. 6A and 6B show an example ofscreen transition in which the object 502 is controlled so as to bemovable in an active range 503 when a contact point is detected in theactive range 503 of the display screen other than the display range ofthe object 502.

FIG. 6A shows a case of detecting a plurality of contact points in afirst coordinate area in which the first display information 501 hasbeen displayed and a second coordinate area in which the second displayinformation 502 has been displayed and detecting, as a scrollinstruction, a change of a detection coordinate of only the contactpoint in the second coordinate area, which is one of the contact points.That is, FIG. 6A shows an operation from Step S401 via “Yes” of StepS402 to “Yes” of Step S403 shown in FIG. 4.

FIG. 6B shows a display result in which the control unit 208 has movedthe second display information (object) 502, which has been displayed inthe coordinate area in which the change of the detection coordinate ofthe contact point has been detected, based upon the change of thedetection coordinate as a result of the identification process of FIG.6A. That is, FIG. 6B shows the operation of Step S404 shown in FIG. 4.

As shown in FIGS. 6A and 6B, when a contact point is detected in theactive range 503 of the display screen other than the display range ofthe object, and a detection coordinate of that contact point does notchange, then the object is controlled so as to be movable within theactive range 503. Accordingly, the object can be moved while theschedule chart is fixed.

When a plurality of contact points are detected in the first coordinatearea and the second coordinate area, and a change of the detectioncoordinate of the contact point in the first coordinate area is detectedwithout detecting a change of the detection coordinate of the contactpoint in the second coordinate area, then the control unit 208 mayscroll (move) only the first display information 501 based upon thechange of the detection coordinate. The contact points are notnecessarily required to be present in both of the first coordinate areaand the second coordinate area.

FIGS. 7A and 7B are diagrams showing another transition example of thedisplay screen in a case of multiple touches. FIGS. 7A and 7B show anexample of screen transition in which an object of “Meeting” iscontrolled so as to be movable with the schedule chart being fixed whena contact is detected at a preset position (designated position,designated coordinate range, movement-fixing coordinate area).

FIG. 7A shows a case of detecting a contact point at a preset position(movement-fixing coordinate area) 504 provided in the first coordinatearea in which the schedule chart has been displayed, and a contact pointin the second coordinate area in which the object has been displayed,and further detecting a change of the detection coordinate of thecontact point in the second coordinate area as a movement instruction.

FIG. 7B shows a display result in which the control unit 208 has movedonly the object, which has been displayed in the coordinate area inwhich the change of the detection coordinate of the contact point hasbeen detected, based upon the change of the detection coordinate as aresult of the identification process of FIG. 7A. That is, FIG. 7B showsthe operation of Step S404 shown in FIG. 4.

As shown in FIGS. 7A and 7B, when a contact point is detected at thepreset position, the object is controlled so as to be movable with theschedule chart being fixed. Accordingly, the object can be moved whilethe schedule chart is fixed. Such a control is suitable for a program ofdisplaying the first display information 501 and the second displayinformation 502 in a combined manner (without display control for eachobject).

FIGS. 8A and 8B are diagrams showing still another transition example ofthe display screen in a case of multiple touches. FIGS. 8A and 8B showan example of screen transition in which only an object is controlled soas to be movable with the schedule chart being fixed when a contact isdetected at a preset position as in the case of FIGS. 7A and 7B. FIGS.8A and 8B differ from FIGS. 7A and 7B in the following points: In FIGS.7A and 7B, the movement-fixing coordinate area is set at a portion ofthe schedule chart in which time is shown. In FIGS. 8A and 8B, amovement-fixing specified position 505 is set in a visible manner at alower part of the active range by an application program.

FIG. 8A shows a case of detecting a contact point in the movement-fixingspecified position 505 provided in the first coordinate area in whichthe schedule chart has been displayed, detecting a contact point in thesecond coordinate area in which the object has been displayed, andfurther detecting a change of the detection coordinate of the detectedpoint in the second coordinate area as a scroll instruction.

FIG. 8B shows a display result in which the control unit 208 has movedonly the object, which has been displayed in the coordinate area inwhich the change of the detection coordinate of the contact point hasbeen detected, based upon the change of the detection coordinate as aresult of the identification process of FIG. 8A.

As shown in FIGS. 8A and 8B, when a contact point is detected at themovement-fixing specified position 505, which has been set to bevisible, the object is controlled so as to be movable with the schedulechart being fixed. Accordingly, the object can be moved while theschedule chart is fixed. Such a control is suitable for a program ofdisplaying a number of objects. Additionally, such a control is suitablefor an application program of displaying display information using alayer structure having a number of layers. When a layer structure isused, the control unit sets the object (second display information) onan upper layer of the schedule chart (first display information).Specifically, the second display information is displayed such that itis overlaid on the first display information.

FIGS. 9A and 9B are diagrams showing still another transition example ofthe display screen in a case of multiple touches. FIGS. 9A and 9B showan example of screen transition in which only an object is controlled soas to be movable with the schedule chart being fixed when a contact isdetected at a preset position as in the other transition examples ofmultiple touches. FIGS. 9A and 9B differ from FIGS. 8A and 8B in thefollowing points: In FIGS. 8A and 8B, the movement-fixing specifiedposition 505 is set at the lower part of the active range by theapplication program for the schedule chart. In FIGS. 9A and 9B, amovement-fixing object 506 is set by a program other than theapplication program for the schedule chart.

FIG. 9A shows a case of detecting a contact point (selection) in thecoordinate area in which the movement-fixing object 506 has beendisplayed, which are provided in the first coordinate area in which theschedule chart has been displayed, detecting a contact point in thesecond coordinate area in which the object to be moved has beendisplayed, and further detecting a change of the detection coordinate ofonly the contact point in the second coordinate area as a scrollinstruction.

FIG. 9B shows a display result in which the control unit 208 has movedonly the object, which has been displayed in the coordinate area inwhich the change of the detection coordinate of the contact point hasbeen detected, based upon the change of the detection coordinate as aresult of the identification process of FIG. 9A.

As shown in FIGS. 9A and 9B, when a contact point is detected in thecoordinate area in which the movement-fixing object 506 has beendisplayed by a program other than an active application program, theobject is controlled so as to be movable with the schedule chart beingfixed. Accordingly, the object can be moved while the schedule chart isfixed.

The movement-fixing object 506 is movable on the screen. Furthermore,the movement-fixing object 506 may be displayed in the foreground. Withsuch a control, assuming that the schedule chart is likened to a paper,the movement-fixing object can be likened to a weight or a paperweight.Thus, a user's intuitive operation can properly be detected, and anobject provided with a movement instruction can properly be moved.

FIGS. 10A and 10B are diagrams showing still another transition exampleof the display screen in a case of multiple touches. FIGS. 10A and 10Bshow an example of screen transition in which only an object iscontrolled so as to be movable with the schedule chart being fixed whena contact is detected at a preset position as in the other transitionexamples of multiple touches. FIGS. 10A and 10B differ from thetransition example of FIGS. 8A and 8B in the following points: In FIGS.8A and 8B, the movement-fixing specified position 505 is set at thelower part of the active range by the application program for theschedule chart. In FIGS. 10A and 10B, an underlay fixing input portion507 is set in an area outside of the display screen, but within adetection range of the display unit 206 having a contact detectionfunction.

FIG. 10A shows a case of detecting contact with the underlay fixinginput portion 507 and contact with the second coordinate area in whichthe object to be moved has been displayed, and further detecting achange of the detection coordinate of only the contact point in thesecond coordinate area as a scroll instruction.

FIG. 10B shows a display result in which the control unit 208 has movedonly the object, which has been displayed in the coordinate area inwhich the change of the detection coordinate of the contact point hasbeen detected, based upon the change of the detection coordinate as aresult of the identification process of FIG. 10A.

As shown in FIGS. 10A and 10B, when contact with the underlay fixinginput portion 507 is detected, the object to be moved is controlled soas to be movable in a state in which the underlay (the schedule chart inthis example) is fixed. Accordingly, the object can be moved while theunderlay is fixed. With such a control, assuming that the schedule chartis likened to a paper, the underlay fixing input portion can be used topress an edge portion of the paper. Thus, a user's intuitive operationcan properly be detected, and an object can properly be moved.

FIGS. 11A and 11B are diagrams showing still another transition exampleof the display screen in a case of multiple touches. FIGS. 11A and 11Bshow an example of screen transition in which only an object iscontrolled so as to be movable with the schedule chart being fixed whena plurality of contact points are concurrently detected as in the othertransition examples of multiple touches. In the example shown in FIGS.11A and 11B, when a user touches a coordinate area corresponding to thedisplay screen frame 500 and the underlay (the schedule chart in thisexample) with a finger of a hand holding the mobile terminal 200 at thesame time, another object is controlled so as to be movable while theunderlay is fixed. A multi-touch screen detectable area, which is anarea of the display unit 206 having a contact detection function fordetecting contact, is provided not only on the display screen, but alsoon the display screen frame. Furthermore, as shown in FIG. 12, a contactpoint continuously extending from the display screen frame 500 to thedisplay screen 509 is deemed contact with the movement-fixing coordinatearea.

FIG. 11A shows a case of detecting a contact point in the coordinatearea corresponding to the display screen frame 500 and the underlay witha finger of a hand holding the mobile terminal 200, detecting a contactpoint in the second coordinate area in which the object to be moved hasbeen displayed, and further detecting a change of the detectioncoordinate of only the contact point in the second coordinate area as ascroll instruction.

FIG. 11B shows a display result in which the control unit 208 has movedonly the object displayed in the coordinate area in which the change ofthe detection coordinate of the contact point has been detected, basedupon the change of the detection coordinate as a result of theidentification process of FIG. 11A.

FIG. 12 is an enlarged view of the display screen shown in FIG. 11. Asshown in FIG. 12, contact with the movement-fixing coordinate area isdetected if a contact point (area) hangs over both of the display screenframe (enclosure) 500 and the display screen 509. Specifically, when thedisplay screen frame (enclosure) 500 and the underlay are brought intocontact with the same finger of the user at the same time, it isdetermined that the operation is for fixing the underlay.

As shown in FIGS. 11A, 11B, and 12, when a user touches the displayscreen frame and the underlay (the schedule chart in this example) witha finger of a hand holding the mobile terminal 200 at the same time, theobject to be moved is controlled so as to be movable in a state in whichthe underlay (the schedule chart in this example) is fixed. Accordingly,the object can be moved while the underlay is fixed. With such acontrol, assuming that the schedule chart is likened to a paper, thepaper can be pressed against the housing of the mobile terminal 200. Inother words, the underlay can be sandwiched between the housing and thefinger. Therefore, a user's intuitive operation can properly beacquired, and an object can properly be moved.

With the control in the illustrated transition examples, when aplurality of contact points are detected, an input item of “Meeting”(second display information) is made movable while the schedule chart(first display information), which is an underlay (background), is fixed(is not allowed to move). Thus, it is possible to detect a user'sintuitive pressing operation, detect a movement (scroll) instruction ofdisplay information from the user's operation, and move only the inputitem.

With such a control, when an input item is to be moved, it is possibleto omit conventional multiple operations, such as selecting the inputitem, displaying the detailed information, and changing and determiningthe detailed information. Unlike a conventional method of moving theinput item by changing time-specifying information included inconventional detailed information, the input item can be moved whileother schedules that have already been inputted are being confirmed.Accordingly, double-booking of schedules can be prevented.

In a case where the object being moved reaches the vicinity of an edgeof the display screen, if information displayed on the edge (edge of thedisplay screen of the display information such as the background or theunderlay) is controlled so as to move around a central portion of theactive range or is controlled so as to move such that movement of theobject is not inhibited, then a user's intuitive operation is furtherfacilitated. For example, in a case where display information needs aplurality of screen displays (a plurality of pages), the control unitmay detect arrival of the object being moved to the vicinity of a loweredge of the display screen and move the information displayed on theedge to the vicinity of a central portion of the active range, or mayswitch the display information into the next page.

From another point of view with regard to the above transition examples,a movement-fixing coordinate area is provided in a first display rangein which first display information is displayed and/or a second displayrange in which second display information is displayed. In a case ofdetecting a first contact point in the movement-fixing coordinate area,detecting a second contact point in either one of the first coordinatearea and the second coordinate area other than the movement-fixingcoordinate area, and detecting a change of the detection coordinate ofthe second contact point, only the first or second display informationdisplayed in the first or second coordinate area in which the secondcontact point is present can be moved based upon the change of thedetection coordinate.

In the above examples, the application program is for the schedulechart. However, the first display information (underlay or background)is not limited to a schedule chart. A map, a calendar, a photographlist, and the like may be displayed as the first display information.Furthermore, the application program may be a document input program ora spreadsheet program. In this case, a selected range of input text oran individual cell can be used as an object. In the case where thepresent invention is applied to a document input program, a selectedpart of text can be moved as an object provided with a movementinstruction in addition to photographs and figures by a user's intuitiveoperation.

Furthermore, in a case where the background display is separated into aplurality of areas, such as a spreadsheet program, only a coordinatearea in which a contact point is present may be controlled to be fixed.In such a case, a predetermined area can be compared with another areaaway from the predetermined area in a state in which those areas areplaced side by side.

Next, a tablet computer 300 according to a third embodiment of thepresent invention will be described.

FIG. 13 is a functional block diagram showing an overview of aconfiguration of the tablet computer 300 according to the presentembodiment.

As shown in FIG. 13, the tablet computer 300 includes a control unit 310operable to perform a variety of calculations, a RAM 311 operable totemporarily store information, a ROM 312 in which a fundamental controlprogram has been stored, a touch panel display 320 operable to input andoutput information, a storage unit 330, and a network interface 313 forcommunication with the Internet or the like via a network.

The touch panel display 320 has a display unit 321 and a contactdetection unit 322. The touch panel display 320 displays displayinformation outputted from the control unit 310, such as images,figures, characters, and symbols, on a display screen of the displayunit 321. The touch panel display 320 is operable to detect an inputfrom a user through contact with the contact detection unit 322.

The storage unit 330 stores therein various application programs 331, adriver 332 for the touch panel display 320, various types of data,various types of contents, an OS (Operating System) 333, and the like.

The control unit 310 operates in accordance with instructions from theOS 333, the driver 332, and the application programs 331. Softwareprograms such as the OS 333 are expanded in the RAM 311 as needed andused.

Furthermore, the contact detection unit 322 is operable to detectcontact with hardware of the contact detection unit 322 (multi-touchscreen detectable area) based upon one of the OS 333, the driver 332,and the application programs 331, or a combination thereof. The contactdetection unit 322 may use any detection methods as long as it candetect a plurality of contact points (areas). Furthermore, the contactdetection unit 322 may be implemented by using any software program. Thecontact detection unit 322 is not specifically limited as long as it candetect contact.

With such a configuration, the tablet computer 300 can properly detect auser's intuitive operation with use of the contact detection unit 322and can properly move an object.

Next, a process of identifying an operation of the tablet computer 300will be described. The identification process will be described in anexample of a spreadsheet program with reference to FIGS. 14, 15A, and15B.

FIG. 14 is a flow chart explanatory of a movement operation of thetablet computer 300.

Processes from S1401 to S1403 shown in FIG. 14 are the same as theprocesses from Step S401 to S403 of FIG. 4. Furthermore, processes ofSteps S1405 and S1406 are the same as the processes of Steps S405 andS406 of FIG. 4.

The control unit 310 of the tablet computer 300 monitors contact withthe touch panel display 320 (Step S1401). If any contact is detected,the control unit 310 judges whether there is one contact point or are aplurality of contact points (Step S1402). Furthermore, the control unit310 detects contact with a display range of another object (anothercell) on the display screen other than a display range of an object tobe moved (cell to be moved) and judges whether a movement instructionfor the object to be moved can be detected (Step S1403).

If contact with the display range of the other object and a movementinstruction for the object to be moved are detected, then the controlunit 310 controls the object to be moved so as to be movable while theother object is fixed (Step S1404).

Meanwhile, in a case where there is one contact point or the like, apredetermined normal operation is performed (Steps S1405-S1406).

Here, a movement operation of an application program in which thebackground (underlay or first display information) is separated into aplurality of areas (objects) will be described with use of a transitionexample of a display screen of a spreadsheet program.

FIGS. 15A and 15B are diagrams showing a transition example of a displayscreen 600 of a spreadsheet program in a case of multiple touches.

FIG. 15A shows a case of detecting a plurality of contact points in afirst coordinate area in which first display information (table in thisexample) 601 has been displayed and in a second coordinate area in whichsecond display information (each cell in this example) 602 has beendisplayed and further detecting a change of a detection coordinate ofonly the contact point in the second coordinate area as a scrollinstruction. That is, FIG. 15A shows an operation from Step S1401 via“Yes” of S1402 to “Yes” of Step S1403 shown in FIG. 14.

FIG. 15B shows a display result in which the control unit 310 hasscrolled only the second display information (object), which has beendisplayed in the coordinate area in which the change of the detectioncoordinate of the contact point has been detected, based upon the changeof the detection coordinate as a result of the identification process ofFIG. 15A. That is, FIG. 15B shows the operation of Step S1404 shown inFIG. 14.

In the screen transition, as shown in FIG. 15B, a cell 604 provided witha movement instruction (cell of 7/10 and 9:00) is controlled so as to bemovable while a rectangular area 603 with corners of the upper-left celland the cell provided with no movement instruction (cell to be fixed) isfixed. Furthermore, if the cell 604 controlled so as to be movable ismoved near an edge of the active area, then the first displayinformation is displayed in a divided manner. The cell 604 to be movedis controlled so as to be movable while a frame portion of thespreadsheet program is displayed.

As shown in FIGS. 15A and 15B, when contact with the active range of thedisplay screen other than the display range of the object provided withthe movement instruction is detected, then the object provided with themovement instruction is controlled so as to be movable within the activerange. Accordingly, the user can move information while he/she views andcompares the cell being moved (7/10 9:00) and other cells away from thatcell being moved with each other in the vertical and horizontaldirections.

In other words, when the control unit 310 detects contact with thecoordinate area in which a plurality of pieces of second displayinformation have been displayed in a frame in which the first displayinformation has been displayed and detects a change of the detectioncoordinate of only one contact point, then the first display informationis displayed in a divided manner. Only the second display informationdisplayed in the coordinate area in which the change of the detectioncoordinate of the contact point has been detected is moved based uponthe change of the detection coordinate.

As described above, an information processing apparatus of the presentinvention recognizes contact of a user with a screen at a plurality ofpoints and an operation directed to a displayed object. Movement controlof an object is performed based upon the recognition result. Therefore,a burden of a user's input operation can be reduced.

Specifically, according to the present invention, there can be providedan information processing apparatus that can properly detect a user'sintuitive operation with use of a contact detection unit capable ofdetecting a plurality of contact points and can properly move an object,a program for implementing such an information processing apparatus, anda computer-readable storage medium in which such a program is recorded.

Furthermore, two or more examples of the aforementioned intuitive fixingoperations may be used in a combined manner. Such a combination canabsorb differences between individual users and further enhance theusability of the users.

Moreover, the specific configuration of the present invention is notlimited to the above embodiments. Any modifications would be included inthe present invention without departing from the spirit of the presentinvention.

This application claims the benefit of priority from Japanese patentapplication No. 2008-185622, filed on Jul. 17, 2008, the disclosure ofwhich is incorporated herein in its entirety by reference.

INDUSTRIAL APPLICABILITY

The present invention is applicable to any information processingapparatus having a contact detection unit capable of detecting aplurality of contact points. Furthermore, the present invention issuitable for an information processing apparatus held by a single handto operate.

1. An information processing apparatus comprising: a display unitoperable to display objects; a contact detection unit operable to detecta plurality of contact points and their positions on the display unit;and a control unit operable to detect change of the positions of thecontact points and to move the objects on the display unit according tothe change of the positions of the contact points, wherein, in a case ofdisplaying a matrix area and at least one of the objects on the matrixarea, and detecting a first contact on one of the objects and change ofa position thereof and a second contact on the matrix area, the controlunit controls the one of the objects to move in range of matrix area. 2.The information processing apparatus as recited in claim 1, wherein, ina case of detecting no contact to the object, the control unit furthercontrols to arrange the object at a position as indicated by the matrixarea.
 3. The information processing apparatus as recited in claim 2,wherein the matrix area is displayed as a plurality of indexes fordetermination of positioning on the matrix area.
 4. The informationprocessing apparatus as recited in claim 1, wherein the control unitsets the object as a visible object displayed on an upper layer of thematrix area or a visible object displayed such that the object isoverlaid on the matrix area.
 5. The information processing apparatus asrecited in claim 2, wherein the control unit sets the object as avisible object displayed on an upper layer of the matrix area or avisible object displayed such that the object is overlaid on the matrixarea.
 6. The information processing apparatus as recited in claim 3,wherein the control unit sets the object as a visible object displayedon an upper layer of the matrix area or a visible object displayed suchthat the object is overlaid on the matrix area.
 7. A computer-readablestorage medium having a program recorded thereon for executing aprocedure with a control unit, the procedure comprising: detecting afirst contact point in a matrix area on a display screen other than adisplay range of an object displayed on a display unit with use of acontact detection unit; detecting a second contact point in the displayrange of the object with use of the contact detection unit; andcontrolling the object so as to be movable within the matrix area when amovement instruction to the object is detected.
 8. An object movementmethod, comprising: detecting a first contact point in a matrix area ona display screen other than a display range of an object displayed on adisplay unit with use of a contact detection unit; detecting a secondcontact point in the display range of the object with use of the contactdetection unit; and controlling the object so as to be movable withinthe matrix area when a movement instruction to the object is detected.